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FOREWORD 



The Quality Measurement Project has had as one of its purposes 
the devising of tools which the school administrator can use to gauge 
the effectiveness of his organization* Such an objective necessitates 
a search for methods of presenting meaningful information about many 
facets of educational programs. Expressing complex interrelationships 
in simple and easily understood terras is an ever present problem. 
Applying nomographic techniques to determine school system effective- 
ness is one solution to this problem. 

Charles Armstrong, a retired staff member of the Quality Measure- 
ment Project, conceived of the application of nomographic techniques 
as a means of determining school system effectiveness. Gerald Wohlferd, 
Associate in Education Research, authored this document. Lee Wolfe, 

Chief of the Bureau of Statistical Services, helped design the nomo- 
graphic charts. 

This document is offered as an illustration of the use of nomo- 
graphic techniques in assessing school system effectiveness. 



LORNE H. WOOLLATT 
Associate Commissioner for 
Research and Evaluation 
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QUALITY EVALUATION THROUGH NOMOGRAPHS 



Introduction 

How good Is a particular school system? Answering this question 
has traditionally been the responsibility of boards of education. 

The superintendent of schools or chief school administrative officer 
has had to evaluate student progress for the board of education. In 
the evaluation of his system the administrator has relied upon both 
subjective and objective measures. Subjective measures have consisted 
of such things as observations of pupils and teachers at work, teacher 
comments, and feedback from parents and pupils. Objective measures 
have often been based upon achievement test results, comparison of 
cost figures, or percent of pupils graduating. Both subjective and 
objective measures, more often than not, have failed to Include re- 
lationships with each other or with other factors. That relationships 
must be considered has been demonstrated by the Quality Measurement 
Project of the New York State Education Department ^ » 12 * ^ 

Such relationships have In the past been presented by the QMP 
In the form of figures and tables in the School Quality Workbook . 

A new method of assessing the quality of a school system, that of 
nomographs, is Illustrated herein. Nomographs are unique in that 
while quality determination Is accomplished In a relatively simple 
manner, relationships are clearly visible In graphic form. 

As nomographs are based upon mathematically derived regression 
equations, a short explanation of regression analysis Is offered for 
the neophyte statistician. An explanation of the structure and use of 
nomographs follows. 




1 



8 



Regression Analysis 



Regression analysis offers a mathematical technique of simul- 
taneously interrelating many quantified measures. Application of 
regression techniques results in a mathematical formula which assigns 
weights, or strengths, to the various measures in relation to each other 
in the prediction of a specific measure. Auxiliary figures indicate 
the accuracy and degree of prediction. 

Regression analysis results can be used in at least two ways in 
education. First, a formula can be derived which provides weights, 
or relative strength indicators for the variables used. Thus, If one 
were to predict an achievement score in reading by using intelligence, 
parental education, and school expenditure meafures, the resultant 
regression equation would indicate the power of each of the three 
predicting measures to each other. 

A second use of regression anal>sis is to compare an actual value 
with a predicted value. Substitution • »t data of an individual school 
district into a general derived equation results in a predicted score 
which can then be compared with the actual score for the district. By 
this method one may determine if children of the district are achieving 
better than expected, as expected, or below expected levels. 

Nomographs 

An extension of regression analysis is the construction of nomo- 
graphic charts. Nomographs express in chart or graphical form the 
relationships expressed as mathematical symbols in the regression 
equation. They combine several graphic scales on a single sheet of 
paper so that reference can be made from one to the other. The value of 
nomographs lies in the simplicity and speed by which information from one 



scale may be translated into information on another scale. For example, 

g 

the formula + 32 = F may be used to convert Fahrenheit temperature 
levels to centrigrade readings or vice versa. Each time one wished 
mathematically to convert from one scale to the other it would be 
necessary to enter a known reading into one side of the formula and 
to the calculate the corresponding reading of the other. Water boils 
at sea level at 100° centrigrade. If one did net already know the 
boiling point on the Fahrenheit scale, the number 100 would need to be 
entered into the above formula in place of C to yield 212° as the 
equivalent Fahrenheit value. The derivation of other equivalent values 
would demand mathematical recalculation for each new situation. 

The relationship between the two temperature scales, expressed 
mathematically above, can also be expressed in a simple nomograph. 

Reference to figure 1, which is such a nomograph, reveals how quickly 
and easily values from the centigrade scale can be translated into 
Fahrenheit values, and vice versa. Equivalent values are directly 
opposite each other on this nomograph. For example, the boiling 
point of the centrigrade scale of 100° is directly opposite and parallel 
to 212° on the Fahrenheit scale. The freezing point of water is 32° 
on the Fahrenheit scale. Its corresponding value on the centigrade 
scale is directly opposite and is easily found to be 0°. The above 
illustrations of the ease and speed of determining relationships is 
ail indication of the facility by which more complex relationships 
can be expressed through the use of nomographs. 

The concept of presenting relationships in nomographic form is 
not new,** but has been used and is being widely used in engineering. I > 2, ^ 
Use in the field of education, ^ however, has been limited. 
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Figure 1. Relationship of centigrade and 
Fahrenheit thermometer scales. 
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Regression analysis was one of the statistical techniques utilized 
by the Quality Measurement Project (QMP).^*^ The construction of nomo- 
graphs based upon the Quality Measurement Project regression analyses is 
a logical step in providing educators with management tools. Accordingly, 
six nomographs (three each for grades 5 and 8) were constructed. In each 
of the nomographs a school system average score on a test of achievement 
is the predicted measure. Average scores were secured through adminis- 
tration of the Iowa Tests of Basic Skills . Form 4.® Grade equivalent 
averages were used. Other data necessary to complete the nomographs are 
mother's education level, father's education level, IQ, and average in- 
structional costs. Directions for compilation of the above factors in 
preparation for entry into the nomographs are to be found in appendixes A 
through D. 

Content of the Nomographs 

No nomograph is a duplicate of any other (see appendix E), The 
scales on each are often unique to that nomograph even though there 
is a general similarity among them. Data incorporated in the columns 
or scales of the six nomographs are distilled in table 1 below. 



Table 1 

Subject Matter Contained in Nomographs 
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Grade 


Nomograph 


Nomograph 


Sublect of Line or Column (from left) 




Number 


Sub iect 


1 


2 


3 


4 


5 


5 


1 


Reading 


IQ 


Line 


Reading 


Instruct . 
Costs 


Mother 1 s 
Educ . 




2 


Arithmetic 


IQ 


Line 


Arithmetic 


Mother* s 
Educ. 


Instruct. 
Cos 1 8 




3 


Composite 


IQ 


Line 


Composite 


Instruct • 
Costs 


Mother 1 s 
Educ. 


8 


4 


Reading 


IQ 


Line 


Mother 1 s 
Educ. 


Reading 


Instruct . 
Costs 




5 


Arithmetic 


Mother' s 
Educ. 


Line 


Father 1 s 
Educ. 


Arithme- 

tic 


IQ 




6 


Composite 


Father 1 s 
Educ. 


Line 


I 


Composite 


Instruct . 
Costs 
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The achievement scales upon which the predicted score will be 
determined are found as the third column from the left margin on 5th 
grade nomographs. The corresponding basic skill scales on 8th grade 
nomographs are the fourth column from the left margin. An unsealed 
column, the use of which will be explained later, appears as column two 
in all nomographs. 

Instructions for Use of Nomographs . Directions for securing 
and preparing the Information dealing with characteristics of the school 
system are to be found in appendixes A-D* As each piece of information 
is derived, it can be entered on the proper scale or scales by an ’X’ 
to represent its location. Lines are then drawn in a designated sequence 
to join the marked locations on the various scales until the predicted 
achievement average score is found where the last line intersects the 
achievement scale. Use of colored pencils to record X's and lines aids 
in distinguishing entries from the basic nomograph content. 

The sequence of connecting scales by lines varies with the 
grade. Each of the columns on the nomographs has been numbered from the 
left across the top of each page. The unsealed vertical line has also 
been numbered. Sequences of connections are shown in table 2. 



Table 2 

Column Connection Sequence 



Grade 


Sequence 




First Connection 


Second Connection 


5 


Column 1 to Column A 


Column 2 to Column 5 


8 


Column 1 to Column 3 


Column 2 to Column 5 
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The procedure to follow is illustrated below for 5th grade read- 
ing score of a hypothetical school system. The school district may have 

found the various measures of their district to be as follows: 

Average IQ (5th Grade) = 110.5 

Average Instructional Costs Per Pupil = $553.2 

Average Mother’s Education (5th Grade) = 3.9 

Average Reading Achievement (5th Grade) = 5.94 

Since the achievement area cited above is 5th grade reading, nomo- 
graph number 1 will be used in the illustration. 

The position of each of the measures is first located on its 
corresponding scale (see figure 2) and noted by an T X f . Scale 1 is then 
joined with scale 4 by a straight line between the X's (see figure 3). 

Next, scale 5 is joined to scale 2 by a straight line. This line 
originates on scale 2, where scale 2 and the line joining scales 1 and 
4 intersect (see figure 4). The predicted achievement score is to be 
found where the last drawn line (between scales 2 and 5) crosses scale 3. 

In the illustrative case the predicted average reading score is approx- 
imately 5.59, while the actual measured score is 5.94. Thus, the 
illustrative school system is averaging +. 35 months above its predicted 
score. 

At the bottom of each sheet containing a nomograph may be found: 
the mathematical formula upon which the nomograph was based, the 
standard error of the predicted measurement , the school year in which 
the majority of the measures were secured, and the number of school 
districts (N) whose data were included in the derivation of the mathematical 
formula. Of particular interest to the user of the accompanying norao- 




7 



14 



1 2 



3 



4 



5 



AVUAOI INITtUCHONAL COSTS 

m run (i> 



AVERAGE TOTAL I.O. 
UOIGE THOINDKI) 






AVIIAr .1 IU 0 INO ACHIEVEMENT 
(WWA TEST) * 



AVUAOI MOTHEI'I EDUCATION 



ft K)UI 01 MOII TEARS 

Of COLLEGE 



— S.J 



S ONE-THIII TEARS 

POST HIGH SCHOOL EDUCATION 



— 4 5 



HIGH SCHOOL OIAOUATI 



— J J 



) n NTH Of MOII lul NOT 

HKJH SCHOOL GIADUATE 



— as 



__ } SEVENTH THIOUGH 
NINTH GIADE 



-II 



I SCXTH OIACE Ol UTS 



— OJ 



0 NO EOIMIL EDUCATION 



NOMOOlAPHI SCHOOL SYSTEM AVUAOI IEADTN 9 AChWImInT, OIADI S 

I* Oil 10 • OOOJMC ♦ 1 P> M ID. ♦ tl 
ITANOAID EllOt ■ Ul 
|as» on ms Data, n. 74 




Figure 2, 



Illustration of entry of values on scales. 
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Illustration of joining of two measures by line. 
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Figure 4. Illustration of completion of nomograph- 
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graphs (appendix E), Is the standard error. The predicted score is 
only as accurate as the measures used in the prediction. The standard 
error describes the accuracy of the predicted score. In the illustra- 
tion the standard error is .303. The actual reading score is about 
3^ months above the predicted score. The difference between 
the actual and predicted scores (.35) is greater than the standard 
error of measurement (.303). Therefore, the conclusion would be 
drawn that the difference is probably a real difference. A difference 
of an actual score from a predicted score which is smaller than the 
standard error, suggests the actual score must be considered as equiv- 
alent to the predicted score. The above statements regarding the 
relationships of differences and standard errors hold true for either 
plus or minus differences. 

Though the above statements are statistically defensible, a word 
of caution is advisable. Since the systems included in the sample may 
not be representative of New York State schools- -New York City was not 
included-- the data on which the regression equations (and subsequently 
the nomographs) were based may be biased. Too, not all elements of 
education which effect achievement in the basic skills hrve been 
included in the equation; as additional important elements are identi- 
fied and their relationships to achievement are determined, accuracy 
of prediction can be increased. Finally, only three achievement measures 
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(reading, arithmetic, and composite) are included in this document. 
Additional achievement areas should be included to make assessment of 
school effectiveness more nearly complete. Logically, other objectives 
of education should also be included in any evaluation of school 
system effectiveness. The nomographs described in this report may 
serve as one approach to assess certain limited areas of a school's 
total program. 
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Appendix A 



Derivation of School System Average 
Achievement Scores 

The scores used in developing the nomographs were obtained from the 
school districts which participated in the Quality Measurement Project 
testing program in the fall of 1965, Each system administered the 
Iowa Tests of Basic Skills, Form 4, 8 to its 5th and 8th grade pupils. 

Children in special class, such as those for the mentally retarded, 
were omitted. Student grade equivalent scores for each subject area 
were averaged by grade for each district. The district averages were 
used to develop the equations used to calculate the predicced scores 
whose scales appear as part of the nomographs. 

School dit .ricts wishing to use the nomographs should use the 
same commercial test. The use of scores obtained by administering 
any other achievement battery is not a valid procedure because of the 
lack of congruence among batteries. As the nomographic scales are 
based upon fall testing, the battery should, if possible, be adminis- 
tered in the fall. If testing is done at any other time of year, the 
averages obtained should be converted to fall equivalents. This is 
accomplished through use of the percentile tables, which appear at 

the rear of the "Manual for Administrators, Supervisors, and Counselors." 8 t 

The average grade equivalent score should be converted to a percentile, 

using the table appropriate for the time of year testing takes place; 

then the percentile should be located in the *beginning-of-year 1 norm 

table where its grade equivalent score may be secured. 



Grade equivalent average scores are expressed in the nomographs 
as decimal fractions with the whole number representing years and 
the decimal as part of the year* Thus, an average grade equivalent 
score of 57.4 which a system might obtain from averaging the grade 
equivalent scores of their 5th grade students' would, in the nomo- 
graph, be expressed as 5.74. This would indicate that the average 
achievement of the students of the system was 5 years and about 7 
months. 



0 

ERIC 



15 



22 



Appendix B 



Directions for Securing Average 
Mother's and Father's Education Level 

Information about the extent of formal education of each child's 
parerics may be secured from the cumulative records, from parent ques- 
tionnaires, or from other sources of data. Large city districts 
may wish to sample their students or parents. Smaller districts may 
find it necessary to include several grades, while very small districts 
will need to include the parents of all students. 

After the educational level has been determined for each parent, 
a value is assigned. Both mother's education and father's education 
are quantified through use of the following scale. 

6 = 4 or more years of college 
5=1-3 years posthigh school education 
4 = High school graduate 

3 = 10th grade or more but not high school graduate 
2 = 7th through 9th grade 
1 = 6th grade or less 
0 = No formal education 

After numbers are assigned for each parent an average is 
obtained by totaling separately for each sex the assigned scale values 
and then dividing by the number of parents of that sex. The resulting 
average educational level is the figure entered on the corresponding 
parental scale of the nomographs . 
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Average mother's education is contained on five of the six 
nomographs, not appearing on monograph 6. Average father's education 
is contained only on nomographs 5 and 6. 
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Appendix C 

Directions for Securing Average IQ Level 

The instrument used to secure pupil IQ was the Lorge- Thorndike 
Intelligence Tests. ^ The total IQ figure was used In construction of 
the nomographs. Use of nonverbal or verbal IQ In the place of total 
IQ Is a questionable procedure. Total IQ scores from a grade 4 
testing may be used If 5th grade scores are unavailable. Similarly, 
7th grade IQ scores may be used under similar clrsumstances in place 
of grade 8 scores. 

Sampling may be utilized for large systems. Pupil scores for 
each grade are averaged as In appendix A. The IQ average score for 
the grade Is then entered Into the nomograph. IQ appears on all six 
nomographs. The average IQ score derived for grade 5 Is entered on 
nomographs 1 through 3. Grade 8 average IQ Is entered on nomographs 
4 through 6. 
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Appendix D 



Directions for Obtaining Average 
Instructional Costs Per Pupil 

The instructional costs which were used in the regression analyses 
and which formed the basis for the scale used in the nomographs were 
drawn from the data of the 1964-65 school year, the year preceding that 
in which achievement testing took place. 

The size of the school system is balanced with its cost by 
expressing costs on a per pupil basis. The school system gross in- 
structional cost figure is account number 296-999 in the Uniform 
S ystem of Accounts fo r School Districts ^ and is entitled, "Instruc- 
tion-Regular Day." This figure is then divided by the "Weighted 
Average Daily Attendance" (WADA). WADA is derived as follows: The 
average of the best four attendance periods is used as the attendance 
figure, with kindergarten pupils counted as pupils in grades 1-6 
counted as 1, and grade 7-12 children counting as lV. Gross instruc- 
tional cost is divided by WADA to obtain the "Average Instructional 
Costs Per Pupil" which is used in all but one of the accompanying 
nomographs. The same figure is entered on all of the c ,e nomo- 
graphs • 
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Nomograph Charts and Condensed Directions 

Directions for completion of nomographs. 

1. Locate positions of measures on each scale with 'X*. 

2. Draw line connecting positions in order shown in Table 2. 

3. Determine predicted score as place where last drawn line 
crosses basic skill achievement column. 

4. Calculate differences of actual scores from predicted 
scores . 

5« Determine if differences are greater than standard errors. 
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